Diverse Evolutionary Neural Networks Based on Information Theory
نویسندگان
چکیده
There is no consensus on measuring distances between two different neural network architectures. Two folds of methods are used for that purpose: Structural and behavioral distance measures. In this paper, we focus on the later one that compares differences based on output responses given the same input. Usually neural network output can be interpreted as a probabilistic function given the input signals if it is normalized to 1. Information theoretic distance measures are widely used to measure distances between two probabilistic distributions. In the framework of evolving diverse neural networks, we adopted information-theoretic distance measures to improve its performance. Experimental results on UCI benchmark dataset show the promising possibility of the approach.
منابع مشابه
A JOINT DUTY CYCLE SCHEDULING AND ENERGY AWARE ROUTING APPROACH BASED ON EVOLUTIONARY GAME FOR WIRELESS SENSOR NETWORKS
Network throughput and energy conservation are two conflicting important performance metrics for wireless sensor networks. Since these two objectives are in conflict with each other, it is difficult to achieve them simultaneously. In this paper, a joint duty cycle scheduling and energy aware routing approach is proposed based on evolutionary game theory which is called DREG. Making a trade-off ...
متن کاملNovel Radial Basis Function Neural Networks based on Probabilistic Evolutionary and Gaussian Mixture Model for Satellites Optimum Selection
In this study, two novel learning algorithms have been applied on Radial Basis Function Neural Network (RBFNN) to approximate the functions with high non-linear order. The Probabilistic Evolutionary (PE) and Gaussian Mixture Model (GMM) techniques are proposed to significantly minimize the error functions. The main idea is concerning the various strategies to optimize the procedure of Gradient ...
متن کاملApplications of Diverse Evolutionary Neural Networks based on Information Theory
Recently, there are many researchers to design neural network architectures using evolutionary algorithms but most of them use the only one fittest solution in the last generation. To exploit much information in the population, an ensemble of the individuals is promising because combining a set of classifiers might produce higher accuracy than the best classifier among them. There is a consensu...
متن کاملEvolving a Cooperative Population of Neural Networks by Minimizing Mutual Information
Evolutionary ensembles with negative correlation learning (EENCL) is an evolutionary learning system for learning and designing neural network ensembles [1]. The fitness sharing used in EENCL was based on the idea of “covering” the same training patterns by shared individuals. This paper explores connection between fitness sharing and information concept, and introduces mutual information into ...
متن کاملOn the convergence speed of artificial neural networks in the solving of linear systems
Artificial neural networks have the advantages such as learning, adaptation, fault-tolerance, parallelism and generalization. This paper is a scrutiny on the application of diverse learning methods in speed of convergence in neural networks. For this aim, first we introduce a perceptron method based on artificial neural networks which has been applied for solving a non-singula...
متن کامل